An Attention Module for Convolutional Neural Networks

Attention mechanism has been regarded as an advanced technique to capture long-range feature interactions and boost the representation capability for convolutional neural networks. However, we found two ignored problems in current attentional activations-based models: approximation problem insufficient capacity of attention maps. To solve together, initially propose module networks by developing AW-convolution, where shape maps matches that weights rather than activations. Our proposed is a complementary method previous attention-based schemes, such those apply explore relationship between channel-wise spatial features. Experiments on several datasets image classification object detection tasks show effectiveness our module. In particular, achieves \(1.00\%\) Top-1 accuracy improvement ImageNet over ResNet101 baseline 0.63 COCO-style Average Precision COCO top Faster R-CNN with backbone ResNet101-FPN. When integrating models, can further increase their up \(0.57\%\) 0.45. Code pre-trained models will be publicly available.